1 Preliminary

1.1 Generalized Binomial Coefficient

For ${\displaystyle a\in \mathbb{C}}$ and any integer ${\displaystyle k\ge 0}$, the generalized binomial coefficient is defined as $$\left(\begin{array}{c}a\\ k\end{array}\right)=\frac{a(a-1)(a-2)\cdots (a-k+1)}{k!}.$$
Specially, if ${\displaystyle a=-m}$ where ${\displaystyle m\in N^{\ast }}$, then $$\left(\begin{array}{c}-m\\ k\end{array}\right)=(-1{)}^k\cdot \left(\begin{array}{c}m+k-1\\ k\end{array}\right).$$

1.2 Newton Binomial Theorem

Let ${\displaystyle a\in \mathbb{C}}$ and ${\displaystyle x\in \mathbb{C}}$ with ${\displaystyle |x|<1}$, then

2 Bernoulli Trial and Bernoulli distribution

A Bernoulli trial is a random experiment having 2 possible outcomes commonly labeled as success and failure. If ${\displaystyle X}$ is the indicator of success, then X follows a Bernoulli distribution or ${\displaystyle X\sim \text{Bernoulli}(p)}$ if $$\mathbb{P}[X=1]=p,\mathbb{P}[X=0]=1-p.$$

2.1 Expectation and variation of Bernoulli distribution

Assume that ${\displaystyle X\sim \text{Bernoulli}(p)}$, therefore $$\begin{array}{rl}& \mathbb{E}[X]=p\\ & \mathbb{E}[X^2]=p\\ & \text{Var}[X]=\mathbb{E}[X^2]-\mathbb{E}[X{]}^2=p(1-p).\end{array}$$

3 Binomial Distribution

3.1 Introduction of binomial distribution

Assume that you are testing the toys manufactured by a factory, where the probability that a toy is defective is ${\displaystyle p}$. In order to decide whether to accept the toys, you randomly sample ${\displaystyle n}$ of them from the whole batch. Let ${\displaystyle X}$ denotes the number of broken toys (${\displaystyle 0\le X\le n}$). Then ${\displaystyle X}$ follows a binomial distribution with parameters ${\displaystyle n}$ and ${\displaystyle p}$ $$X\sim \text{Bin}(n,p).$$ The pmf of ${\displaystyle X}$ is $$\mathbb{P}[X=k]=\left(\begin{array}{c}n\\ k\end{array}\right)p^k(1-p{)}^{n-k}k\in \{0,1,\cdots ,n\}.$$

3.2 Expectation and Variation of binomial distribution

1. We can directly compute the expectation and variation by calculating the first and second moment of ${\displaystyle X}$. $$\begin{array}{rl}\mathbb{E}[X]& =\sum _{k=0}^{n}k\cdot \left(\begin{array}{c}n\\ K\end{array}\right)p^k(1-p{)}^{n-k}\\ & =\sum _{k=1}^{n}k\cdot \left(\begin{array}{c}n\\ K\end{array}\right)p^k(1-p{)}^{n-k}\\ & =np\sum _{k=1}^n\left(\begin{array}{c}n-1\\ K-1\end{array}\right)p^{k-1}(1-p{)}^{(n-1)-(k-1)}\\ & =np\sum _{t=0}^{n-1}\left(\begin{array}{c}n-1\\ T\end{array}\right)p^t(1-p{)}^{n-1-t}=np.\end{array}$$

therefore $$\begin{array}{rl}& \mathbb{E}[X]=np\\ & \text{Var}[X]=\mathbb{E}[X^2]-\mathbb{E}[X{]}^2=np(1-p).\end{array}$$
2. Let ${\displaystyle {\xi }_i}$ denote the indicator variable of success of the ${\displaystyle i}$th Bernoulli trial. Then the number of successes can be expressed as ${\displaystyle X={\xi }_1+\cdots +{\xi }_n}$. Using the expectation and variation of Bernoulli distribution and linearity of expectation and variation for independent random variables. We get $$\begin{array}{rl}& \mathbb{E}[X]=\mathbb{E}[{\xi }_1+\cdots +{\xi }_n]=\sum _{i=1}^n\mathbb{E}[{\xi }_i]=np\\ & \text{Var}[X]=\text{Var}[{\xi }_1+\cdots +{\xi }_n]=\sum _{i=1}^n\text{Var}[{\xi }_i]=np(1-p).\end{array}$$

4 Geometric Distribution

4.1 Pmf of Geometric Distribution

The geometric distribution gives the probability that the first occurrence of success requires ${\displaystyle k}$ independent trials, each with probability ${\displaystyle p}$. Assume a random variable ${\displaystyle X\sim \text{Geo}(p)}$, then $$\mathbb{P}[X=k]=(1-p{)}^{k-1}p,k=1,2,3,\cdots .$$

4.2 Expectation of Geometric Distribution

Here we use an interesting trick to solve ${\displaystyle \mathbb{E}[X]}$ where ${\displaystyle X\sim \text{Geo}(p)}$. $$\begin{array}{rl}\mathbb{E}[X]& =\sum _{k=1}^{\mathrm{\infty }}k\cdot (1-p{)}^{k-1}p\\ & =p\sum _{k=1}^{\mathrm{\infty }}[(q{)}^k{]}^{\prime }\\ & =p\cdot {\left(\frac{q}{1-q}\right)}^{\prime }\\ & =\frac{1}{p}.\end{array}$$

4.3 Variation of Geometric Distribution

4.4 Sum of Geometric-distributed random variables

Assume we have ${\displaystyle n}$ random variables ${\displaystyle X_1,\cdots ,X_n}$ satisfying ${\displaystyle X_i\sim \text{Geo (p)}}$, ${\displaystyle i=1,\cdots ,n}$. From the definition of Geometric distribution we know that it is about the number of trials when the first success occurs. Then ${\displaystyle Y=X_1+X_2+\cdots +X_n}$ indicates the number of trials when the ${\displaystyle n\text{th}}$ success occurs. Therefore the last trial has succeeded and the remaining trials have ${\displaystyle n-1}$ successes. Thus $$\begin{array}{rl}\mathbb{P}[Y=k]& =p\cdot \left(\begin{array}{c}k-1\\ n-1\end{array}\right)p^{n-1}(1-p{)}^{k-n}\\ & =\left(\begin{array}{c}k-1\\ n-1\end{array}\right)p^n(1-p{)}^{k-n}.\end{array}$$We call this distribution Negative Binomial Distribution.

5 Pascal Distribution

In ${\displaystyle n}$ independent Bernoulli trials with success probability ${\displaystyle p}$, the pascal distribution focuses on the number of failures when the ${\displaystyle r}$th success happens. If a random variable ${\displaystyle X\sim \text{Pascal}(r,p)}$, then $$\mathbb{P}[X=k]=\left(\begin{array}{c}k+r-1\\ k\end{array}\right)p^r(1-p{)}^k,k=0,1,2,\cdots .$$

5.1 Expectation and variation of Pascal distribution

If ${\displaystyle X\sim \text{Pascal}(r,p)}$, then $$\mathbb{E}[X]=\frac{r(1-p)}{p},\text{Var}[X]=\frac{r(1-p)}{p^2}.$$

6 Negative Binomial Distribution

In ${\displaystyle n}$ independent Bernoulli trials with success probability ${\displaystyle p}$, the negative binomial distribution focuses on the number of trials when the ${\displaystyle r}$th success happens. If a random variable ${\displaystyle X\sim \text{NB}(r,p)}$, then $$\mathbb{P}[X=k]=\left(\begin{array}{c}k-1\\ r-1\end{array}\right)p^r(1-p{)}^{k-r},k=r,r+1,\cdots .$$From the above discussion about geometric distribution, we have ${\displaystyle X=X_1+X_2+\cdots X_r}$ where ${\displaystyle X_i\sim \text{Geo}(p)(i\in [r])}$, where ${\displaystyle X_i}$ denotes the number of trials needed after the ${\displaystyle (i-1)}$st success to obtain the ${\displaystyle i}$th success. (${\displaystyle X_1,\cdots ,X_r}$ are independent)

6.1 Expectation and variation of negative binomial distribution

7 Gaussian Distribution

7.1 Pdf of Gaussian Distribution

If a random variable ${\displaystyle X\sim \mathcal{N}(\mu ,{\sigma }^2)}$, where ${\displaystyle \mu }$ is the mean and ${\displaystyle {\sigma }^2}$ is the variance, then the pdf of ${\displaystyle X}$ is $$f(x)=\frac{1}{\sqrt{2\pi }\sigma }\exp \{-\frac{(x-\mu {)}^2}{2{\sigma }^2}\}.$$

7.2 Mgf of Gaussian Distribution

Assume a random variable ${\displaystyle X\sim \mathcal{N}(\mu ,{\sigma }^2)}$, then the mgf of ${\displaystyle X}$ is $$M_X(t)=\mathbb{E}[e^{tX}]=\exp (\mu t+\frac{1}{2}{\sigma }^2{t}^2).$$

7.3 Characteristic function of Gaussian Variables

Since we know the mgf of Gaussian variable ${\displaystyle X\sim N(\mu ,\sigma )}$ is $$M_X(t)=\mathbb{E}[e^{tX}]=\exp (\mu t+\frac{1}{2}{\sigma }^2{t}^2),$$therefore the characteristic function of ${\displaystyle X}$ is equal to $${\phi }_X(t)=\mathbb{E}[e^{itX}]=M_X(it)=\exp (i\mu t-\frac{1}{2}{\sigma }^2{t}^2).$$

7.4 Moments of standard Gaussian

Assume that the random variable ${\displaystyle X\sim N(0,1)}$ and that ${\displaystyle k\in N^{\ast }}$, then $$\mathbb{E}[X^k]=\{\begin{array}{ll}0& k\text{ odd}\\ (k-1)!!& k\text{ even}.\end{array}$$
First we state an important lemma:

Since we have the mgf of Gaussian variables, moments can be obtained via differentiation. We also provide a direct proof for the moments of the standard Gaussian.

If ${\displaystyle k}$ is odd, then it's obvious that ${\displaystyle \mathbb{E}[X^k]}$=0, so the following process assumes that ${\displaystyle k}$ is even. Then $$\begin{array}{rl}\frac{1}{\sqrt{2\pi }}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{x}^k{e}^{-\frac{x^2}{2}}\mathrm{d}x& =\sqrt{\frac{2}{\pi }}{\int }_0^{\mathrm{\infty }}{x}^k{e}^{-\frac{x^2}{2}}\mathrm{d}x\\ & =\sqrt{\frac{2}{\pi }}{\int }_0^{\mathrm{\infty }}{t}^{\frac{k-1}{2}}{e}^{-\frac{t}{2}}\mathrm{d}t\\ & =\sqrt{\frac{2}{\pi }}\cdot \frac{1}{2}\cdot \frac{\mathrm{\Gamma }(\frac{k+1}{2})}{{\left(\frac{1}{2}\right)}^{(k+1)\text{/}2}}\\ & =\frac{1}{\sqrt{2\pi }}{2}^{\frac{k+1}{2}}\cdot \frac{(k-1)!!}{2^{\frac{k}{2}}}\cdot \mathrm{\Gamma }(\frac{1}{2})\\ & =(k-1)!!(\text{Recall that }\mathrm{\Gamma }(\frac{1}{2})=\sqrt{\pi }).\end{array}$$

7.5 An important property of Standard Gaussian and Mills Ratio

A standard Gaussian variable ${\displaystyle X\sim N(0,1)}$ has a good property in terms of derivatives that $${\varphi }^{\prime }(x)=-x\varphi (x).$$

7.6 Expectation of absolute value of Gaussian variables

If a random variable ${\displaystyle Z\sim N(0,1)}$, $$\mathbb{E}[|Z|]=\sqrt{\frac{2}{\pi }}.$$

7.7 Rotational Invariance of Gaussian Variables

If a matrix ${\displaystyle R\in {\mathbb{R}}^{n\times n}}$ is orthogonal, indicating that ${\displaystyle R{R}^{\mathrm{\top }}=R^{\mathrm{\top }}R=I_n}$, given a random vector ${\displaystyle X\sim \mathcal{N}(0,{\sigma }^2{I}_n)}$, it satisfies $$RX\stackrel{d}{=}Y.$$This can be proved using linear transformation of multivariate Gaussian variables.

8 Multivariate Gaussian Distribution

8.1 Definition of Standard Normal Random Vector

A real random vector ${\displaystyle X=(X_1,\cdots ,X_k{)}^{\mathrm{\top }}}$ is called a standard normal random vector if all of its components ${\displaystyle X_i}$ (${\displaystyle i\in [k]}$) are independent standard Gaussian variables. We denote it ${\displaystyle X\sim \mathcal{N}(0,I_n)}$ where the mean vector is ${\displaystyle \overrightarrow{0}}$ and the covariance matrix is ${\displaystyle I_n}$.

8.2 Definition of Normal random vector

A real random vector ${\displaystyle X=(X_1,\cdots ,X_k{)}^{\mathrm{\top }}}$ is called a normal random vector if there exists a random normal random vector ${\displaystyle Z\in {\mathbb{R}}^l}$, a ${\displaystyle k\times l}$ matrix ${\displaystyle A}$ and a ${\displaystyle k}$-dim vector ${\displaystyle \mathit{\mu }}$, such that ${\displaystyle X=AZ+\mathit{\mu }}$. (Wikipedia) We denote it ${\displaystyle X\sim \mathcal{N}(\mathit{\mu },\mathrm{\Sigma })}$, where ${\displaystyle \mathit{\mu }}$ is the mean vector and ${\displaystyle \mathrm{\Sigma }}$ is the covariance matrix with $$\mathit{\mu }=\left(\begin{array}{c}{\mu }_1\\ {\mu }_2\\ ⋮\\ {\mu }_k\end{array}\right),\mathrm{\Sigma }=\left(\begin{array}{cccc}{\sigma }_1^2& {\rho }_{12}{\sigma }_1{\sigma }_2& \cdots & {\rho }_{1k}{\sigma }_1{\sigma }_k\\ {\rho }_{21}{\sigma }_2{\sigma }_1& {\sigma }_2^2& \cdots & {\rho }_{2k}{\sigma }_2{\sigma }_k\\ ⋮& ⋮& \cdots & ⋮\\ {\rho }_{k1}{\sigma }_k{\sigma }_1& {\rho }_{k2}{\sigma }_k{\sigma }_2& \cdots & {\sigma }_k^2\end{array}\right),$$where ${\displaystyle {\mathrm{\Sigma }}_{ij}}$ is the covariance between ${\displaystyle X_i}$ and ${\displaystyle X_j}$ (${\displaystyle i,j\in [k]}$) and ${\displaystyle {\mu }_i}$ is the mean of ${\displaystyle X_i}$ (${\displaystyle i\in [k]}$).

8.3 Joint pdf of multivariate Gaussian distribution

Assume that ${\displaystyle X\in {\mathbb{R}}^k\sim \mathcal{N}(\mathit{\mu },\mathrm{\Sigma })}$, then $$f_X(x_1,\cdots ,x_k)=\frac{1}{\sqrt{(2\pi {)}^k|\mathrm{\Sigma }|}}\exp \{-\frac{1}{2}(X-\mathit{\mu }{)}^{\mathrm{\top }}{\mathrm{\Sigma }}^{-1}(X-\mathit{\mu })\}.$$

8.4 Characteristic function of multivariate Gaussian variables

If ${\displaystyle X\in {\mathbb{R}}^k\sim \mathcal{N}(\mathit{\mu },\mathrm{\Sigma })}$, then $${\phi }_X(\mathbf{t})=\mathbb{E}[e^{i{\mathbf{t}}^{\mathrm{\top }}X}]=\exp \{i{\mathbf{t}}^{\mathrm{\top }}\mathit{\mu }-\frac{1}{2}{\mathbf{t}}^{\mathrm{\top }}\mathrm{\Sigma }\mathbf{t}\},\mathbf{t}\in {\mathbb{R}}^k.$$

8.5 Linear Transformation of multivariate Gaussian variables

If ${\displaystyle X\sim \mathcal{N}(\mu ,\mathrm{\Sigma })}$ and ${\displaystyle Y=\alpha +AX}$, then $$\begin{array}{rl}\mathbb{E}[Y]& =A\mu +\alpha \\ \text{Var}[Y]& =A\mathrm{\Sigma }{A}^{\mathrm{\top }}.\end{array}$$

9 Chi-squared Distribution

9.1 Definition of Chi-squared Distribution

Assume that we have ${\displaystyle n}$ ${\displaystyle \text{i.i.d.}}$ samples ${\displaystyle X_1,\cdots ,X_n}$ from ${\displaystyle N(0,1)}$. Then $$\sum _{i=1}^n{X}_i^2\sim {\chi }_n^2,$$which is called Chi-squared Distribution with ${\displaystyle n}$ degrees of freedom.

9.2 Pdf of Chi-squared Distribution

From the first part, if ${\displaystyle X\sim N(0,1)}$, then ${\displaystyle Y=X^2\sim {\chi }_1^2}$. Then $$\begin{array}{rl}{F}_Y(y)=\mathbb{P}[X^2\le y]& =\mathbb{P}[-\sqrt{y}\le X\le \sqrt{y}]\\ & ={\int }_{-\sqrt{y}}^{\sqrt{y}}\frac{1}{\sqrt{2\pi }}{e}^{-\frac{t^2}{2}}\mathrm{d}t.\end{array}$$Let ${\displaystyle G(t)}$ denote the primitive function of ${\displaystyle \frac{1}{\sqrt{2\pi }}\exp \{-\frac{t^2}{2}\}}$. Therefore $$\begin{array}{rl}{f}_Y(y)=F_Y^{\prime }(y)& =(G(\sqrt{y})-G(-\sqrt{y}){)}^{\prime }\\ & =\frac{1}{\sqrt{2\pi }}{e}^{-\frac{y}{2}}\frac{1}{\sqrt{y}}\\ \text{(1)}& & =\frac{{\left(\frac{1}{2}\right)}^{1\text{/}2}}{\mathrm{\Gamma }(\frac{1}{2})}{y}^{\frac{1}{2}-1}{e}^{-\frac{1}{2}y}\end{array}$$where ${\displaystyle (1)}$ is because ${\displaystyle \mathrm{\Gamma }(\frac{1}{2})=\sqrt{\pi }}$.

9.3 Relation to Gamma Distribution

In the derivation of the pdf of Chi-squared Distribution we can conclude that $${\chi }_1^2\iff \mathrm{\Gamma }(\frac{1}{2},\frac{1}{2}),$$which is a special case of Gamma distribution. Therefore, if a random variable ${\displaystyle X\sim {\chi }_n^2}$, then ${\displaystyle X\sim \mathrm{\Gamma }(\frac{n}{2},\frac{1}{2})}$.

9.4 Expectation of Chi-squared Distribution

Given that if a random variable ${\displaystyle Y\sim \mathrm{\Gamma }(\alpha ,\beta )}$, then ${\displaystyle \mathbb{E}[Y]=\frac{\alpha }{\beta }}$, therefore for a random variable ${\displaystyle X\sim {\chi }_n^2}$, $$\mathbb{E}[X]=n.$$

9.5 Variation of Chi-squared Distribution

Given that if a random variable ${\displaystyle Y\sim \mathrm{\Gamma }(\alpha ,\beta )}$, then ${\displaystyle \text{Var}[Y]=\frac{\alpha }{{\beta }^2}}$, therefore for a random variable ${\displaystyle X\sim {\chi }_n^2}$, $$\text{Var}[X]=2n.$$

9.6 Mgf of Chi-squared Distribution

Given that for a Gamma distributed random variable ${\displaystyle X\sim \mathrm{\Gamma }(\alpha ,\beta )}$, the mgf is $$M_X(t)=\frac{1}{(1-\frac{t}{\beta }{)}^{\alpha }},$$since ${\displaystyle Y\sim {\chi }_n^2}$ is equivalent to ${\displaystyle Y\sim \mathrm{\Gamma }(\frac{n}{2},\frac{1}{2})}$, therefore $$M_Y(t)=\frac{1}{(1-2t{)}^{\frac{n}{2}}}.$$

9.7 Characteristic function of Chi-squared Distribution

Since for ${\displaystyle Y\sim {\chi }_n^2}$, ${\displaystyle M_Y(t)=(1-2t{)}^{-\frac{n}{2}}}$, therefore $${\phi }_Y(t)=M_Y(it)=(1-2it{)}^{-\frac{n}{2}}.$$

9.8 Asymptotic Property of Chi-squared Distribution

9.8.1 LLN

From the definition of Chi-squared Distribution we have $$Y=\sum _{i=1}^n{X}_i^2\sim {\chi }_n^2,X_i\sim N(0,1).$$If we treat every ${\displaystyle X_i^2}$ as an ${\displaystyle \text{i.i.d.}}$ sample from the same distribution with mean ${\displaystyle \mathbb{E}[X_i^2]=1}$. By the law of large number we can derive $$\frac{\sum _{i=1}^n{X}_i^2}{n}\stackrel{\mathbb{P}}{\to }\mathbb{E}[X_i^2]=1,$$thus $$\frac{Y}{n}\stackrel{\mathbb{P}}{\to }1,Y\sim {\chi }_n^2.$$

9.8.2 CLT

Similar to the LLN part, since we treated every ${\displaystyle X_i^2}$ as an ${\displaystyle \text{i.i.d.}}$ sample, therefore we can apply Central Limit Theorem to get convergence in distribution:$$\frac{Y-n}{\sqrt{2n}}\stackrel{L}{\to }N(0,1),Y\sim {\chi }_n^2.$$

10 Exponential Distribution

10.1 Pdf of Exponential Distribution

If a random variable ${\displaystyle X\sim \text{Exp}(\lambda )}$, then the pdf of ${\displaystyle X}$ is $$f(x)=\{\begin{array}{ll}\lambda e^{-\lambda x}& x>0\\ 0& \text{elsewhere}.\end{array}$$

10.2 The tail probability of Exponential Distribution

If ${\displaystyle X\sim \text{Exp}(\lambda )}$ with ${\displaystyle \lambda >0}$, then the tail probability is $$\mathbb{P}[X\ge t]=e^{-\lambda t}.$$

10.3 Memoryless Property

10.4 Relation to Gamma Distribution

Exponential Distribution is a special form of Gamma Distribution, using shape-rate version we can find that if ${\displaystyle X\sim \mathrm{\Gamma }(1,\alpha )}$:$$f(x)=\frac{\alpha }{\mathrm{\Gamma }(1)}{x}^{1-1}{e}^{-\alpha x}=\alpha e^{-\alpha x},x>0.$$Therefore $$X\sim \text{Exp}(\lambda )\iff X\sim \mathrm{\Gamma }(1,\lambda ).$$

11 Poisson Distribution

11.1 Pmf of Poisson Distribution

Assume random variable ${\displaystyle X\sim \text{Poisson}(\lambda )}$, the pmf of ${\displaystyle X}$ is given as follows:$$\mathbb{P}[X=k]=\frac{e^{-\lambda }{\lambda }^k}{k!}\lambda >0,k=0,1,2,\cdots .$$

11.2 Expectation and variance of Poisson Distribution

First we have an important property of Poisson Distribution:$$\text{Assume }X\sim \text{Poisson}(\lambda )\text{, then }\mathbb{E}[X(X-1)\cdots (X-k)]={\lambda }^{k+1},k=0,1,2,\cdots$$Using this property we can quickly get that $$\begin{array}{rl}\mathbb{E}[X]& =\lambda \\ \mathbb{E}[X^2]& ={\lambda }^2+\lambda ,\end{array}$$therefore $$\text{Var}[X]=\mathbb{E}[X^2]-\mathbb{E}[X{]}^2=\lambda .$$

11.3 Mgf of Poisson Distribution

If a random variable ${\displaystyle X\sim \text{Poisson}(\lambda )}$ where ${\displaystyle \lambda >0}$, then the mgf of ${\displaystyle X}$ is $$M_X(t)=\exp \{\lambda (e^t-1)\}.$$

11.4 Reproducibility of Poisson distribution

If ${\displaystyle X_1,\cdots ,X_n}$ are independent variables satisfying ${\displaystyle X_i\sim \text{Poisson}({\lambda }_i)}$, then $$\sum _{i=1}^n{X}_i\sim \text{Poisson}(\sum _{i=1}^n{\lambda }_i).$$

11.5 Poisson approximation to the Binomial distribution

For ${\displaystyle n}$ Bernoulli trials with probability ${\displaystyle p}$, if ${\displaystyle n}$ is large enough and ${\displaystyle p}$ is small enough, then the Binomial distribution is approximately equal to Poisson distribution with parameter ${\displaystyle \lambda =np}$.

12 Gamma Distribution

12.1 ${\displaystyle {\displaystyle \mathrm{\Gamma }}}$ function

The Gamma function ${\displaystyle \mathrm{\Gamma }(\cdot )}$ is defined as follows:

An integration by parts shows that$$\mathrm{\Gamma }(\alpha )=(\alpha -1){\int }_0^{\mathrm{\infty }}{y}^{\alpha -2}{e}^{-y}dy=(\alpha -1)\mathrm{\Gamma }(\alpha -1)$$Given that $$\mathrm{\Gamma }(1)={\int }_0^{\mathrm{\infty }}{e}^{-y}dy=1$$Thus if ${\displaystyle \alpha }$ is a positive integer greater than 1, $$\mathrm{\Gamma }(\alpha )=(\alpha -1)!$$

12.2 ${\displaystyle {\displaystyle \mathrm{\Gamma }(\alpha ,\beta )}}$ Distribution (shape-scale version)

We say that the continuous random variable ${\displaystyle X}$ has a ${\displaystyle \mathrm{\Gamma }-\text{distribution}}$ with parameters ${\displaystyle \alpha >0}$ and ${\displaystyle \beta >0}$ if its pdf is $$f(x)=\{\begin{array}{ll}\frac{1}{\mathrm{\Gamma }(\alpha ){\beta }^{\alpha }}{x}^{\alpha -1}{e}^{-x/\beta }& 0<x<\mathrm{\infty }\\ 0& \text{elsewhere},\end{array}$$we often write that ${\displaystyle \mathrm{\Gamma }(\alpha ,\beta )}$ distribution where ${\displaystyle \alpha }$ is the shape parameter and ${\displaystyle \beta }$ is the scale parameter.

12.3 ${\displaystyle {\displaystyle \mathrm{\Gamma }(\alpha ,\beta )}}$ Distribution (shape-rate version)

We say that the continuous random variable ${\displaystyle X}$ has a ${\displaystyle \mathrm{\Gamma }-\text{distribution}}$ with parameters ${\displaystyle \alpha >0}$ and ${\displaystyle \beta >0}$ if its pdf is $$f(x)=\{\begin{array}{ll}\frac{{\beta }^{\alpha }}{\mathrm{\Gamma }(\alpha )}{x}^{\alpha -1}{e}^{-\beta x}& 0<x<\mathrm{\infty }\\ 0& \text{elsewhere},\end{array}$$we often write that ${\displaystyle \mathrm{\Gamma }(\alpha ,\beta )}$ distribution where ${\displaystyle \alpha }$ is the shape parameter and ${\displaystyle \beta }$ is the rate parameter. Throughout this article we use this version of Gamma distribution.

12.4 Expectation of ${\displaystyle {\displaystyle \mathrm{\Gamma }(\alpha ,\beta )}}$ Distribution

We can use the definition of the Gamma function to simplify the computation of the integral:$$\begin{array}{rl}\mathbb{E}[X]& ={\int }_0^{\mathrm{\infty }}\frac{{\beta }^{\alpha }}{\mathrm{\Gamma }(\alpha )}{x}^{\alpha -1}{e}^{-\beta x}\cdot x\mathrm{d}x\\ & ={\int }_0^{\mathrm{\infty }}\frac{{\beta }^{\alpha }}{\mathrm{\Gamma }(\alpha )}{x}^{\alpha }{e}^{-\beta x}\mathrm{d}x\\ & =\frac{{\beta }^{\alpha }}{\mathrm{\Gamma }(\alpha )}\cdot \frac{\mathrm{\Gamma }(\alpha +1)}{{\beta }^{\alpha +1}}{\int }_0^{\mathrm{\infty }}\frac{{\beta }^{\alpha +1}}{\mathrm{\Gamma }(\alpha +1)}{x}^{\alpha }{e}^{-\beta x}\mathrm{d}x\\ & =\frac{\alpha }{\beta }.\end{array}$$